The following example has been tested to work in Internet Explorer 4 to 11.
WebM can achieve the same quality as MP4 with a smaller file size.
Internet Explorer 4 to 7 only support AVI.
The reccomendation is to include an OGG source but this is not widely supported and doesn't seem to provide any benefit over WebM or MP4.
<video width="320" height="240" controls="controls">
<source src="movie.ogg" type="video/webm" />
<source src="movie.mp4" type="video/mp4" />
<embed type="video/x-msvideo" src="movie.avi" width="320" height="240" autostart="false" />
</video>